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DETAILED ACTION 

1. This action is responsive to communications: IDS's received 1/27/2004 and 2/2/04, and 
Amendment C filed 1/27/2004. 

2. Claims 1-34 are pending in the case. Claims 1, 7, 10, 13, 20, 25 and 29 are independent 
claims. 

Claim Rejections - 35 USC § 101 

3. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

4. Claims 10-12 and 25-27 remain rejected under 35 U.S.C- 101 because the claimed 
invention is directed to non-functional descriptive data stored on a medium. 

The claims are not directed to statutory subject matter because the claimed subject 
matter: Does not fall within one of the four statutory classes of inventions under 101; and/or 
falls, by analogy, within, the printed matter exception to 101, 

Data structures do not fall within one of the four statutory classes of invention under 
101: process, machine, manufacture, and composition of matter. A data structure is clearly 
neither a "process" nor a "machine." With regard to the other statutory classes, the Supreme 
Court in Diamond v. Chakrabartv , 206 USPQ 193 (S. Ct. 1980), has defined a "manufacture" as 
"the production of articles for use from raw materials prepared by giving to these materials new 
forms, qualities, properties, or combinations whether by hand labor or by machinery" and has 
defined a "composition of matter" as "all compositions of two or more substances and ... all 
composite articles, whether they be results of chemical union, or of mechanical mixture, or 
whether they be gases, fluids, powders, or solids." Id at 195-196. Clearly, a data structure, per 
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se, cannot be considered a "manufacture" since a data structure is not produced from raw 
materials and has no tangible, physical form or structure. Likewise, a data structure cannot be 
considered a "composition of matter" since a data structure is not a composition of substances or 
composite articles as contemplated by the Supreme Court. Accordingly, since a data structure 
does not fall within one of the four statutory classes of inventions under 101, the claims are not 
directed toward statutory subject matter. 

Lastly, a data structure is considered non-statutory subject matter by analogy to the 
"printed matter" exception under 101. See In re Miller , 164 USPO 46, 49 (CCPA 1969). Like 
printed matter, a data structure, in and of itself, is merely an arrangement of data and nothing 
more. Furthermore, claims drawn to printed matter may be non-statutory even though the claims 
recite the structure on which the printed matter is printed: 

The mere arrangement of printed matter on a sheet or sheets of paper, in book 
form or otherwise, does not constitute "any new and useful art, machine, 
manufacture, or composition of matter," or "any new and useful improvements 
thereof," as provided in section 4886, of the Revised Statutes [the predecessor to 
35U.S.C 101]. 

(emphasis in original). In re Russell 9 USPQ 181, 182 (CCPA 1931). At best, the claims as a 
whole describe a data structure stored in a computer system. Accordingly, like printed matter 
"stored" on a sheet of paper, a data structure stored in a computer system fails to present 
statutory subject matter. 
MPEP§2106 states: 

When nonfunctional descriptive material is recorded on some computer-readable medium, it 
is not statutory since no requisite functionality is present to satisfy the practical application 
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requirement. Merely claiming nonfunctional descriptive material stored in a computer- 
readable medium does not make it statutory. Such a result would exalt form over substance. 
In re Sarkar, 588 F.2d 1330, 1333, 200 USPQ 132, 137 (CCPA 1978) ("[E]ach invention 
must be evaluated as claimed; yet semantogenic considerations preclude a determination 
based solely on words appearing in the claims. In the final analysis under 101, the claimed 
invention, as a whole, must be evaluated for what it is.") (quoted with approval in Abele, 684 
F.2d at 907, 214 USPQ at 687). See also In re Johnson, 589 F,2d 1070, 1077, 200 USPQ 
199, 206 (CCPA 1978) ("form of the claim is often an exercise in drafting"). 



Although the independent claim recites "data structure" in the preamble, the body of the 
claim merely recites a collection of fields. Use of the broad phrase "correlated through" is 
insufficient to establish the necessary functional interrelationship between elements to constitute 
a true data structure. As such, the claims recite non- functional descriptive data stored on a 
medium. As per the claims recite "non functional" data stored in a computer readable medium, 
and thus are not considered statutory, since the non-functional data is mere arrangement of data. 



Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-12, 20, 22, 23 and 25-28 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Synchronized Multimedia Integration Language (SMIL) 1.0 
Specification," W3C Recommendation 15-June-1998, herein referred to as W3C, in view of 
deVries et al. (USPN 6332144 — (priority date 3/11/1998). 
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Regarding independent claim 1, W3C discloses a system (p.26, ex. 4) for associating 
annotations, contained in the <text ... /> tag, to the plurality of media streams, representing 
different versions of multimedia content, that would be contained in ellipses found at lines 6, 9, 
and 12 of the example, as explained in the header of the example. The text is contained with a 
<par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as noticed in the 
example's text, are "alternative parts designed for screens with different resolutions, and 
bit-depths." The ellipses at 6, 9, and 12, are clearly meant to be the alternative parts. These are 
the different versions of multimedia content. They would be represented by one of the tags 
disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can be 
associated to particular time frames (p. 12). As the text is assigned to a particular "temporal 
location in the multimedia presentation," it is equivalent to an annotation as defined in the 
specification, (page 2, line 19 - page 3, line 7), even meeting the specific example of "temporal 
location." 

While there is no explicit mention of multimedia servers, W3C does however disclose that both 
the annotations and media are referenced by a URI (p. 19, "src"), which were well-known to 
refer to servers. deVries et al. (deVries) discloses that media and annotations can be stored on 
servers (FIG IB, 22 and 26), the annotation server inherently having a database of annotations. 
Therefore it would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine deVries and W3C in order to effectively store the annotations (col, 14, 11. 
5-8), media (col. 4, 11. 34-08), and relations between the two (FIG. 7). 

Regarding dependent claim 2, deVries discloses the streams can be audio or video (col. 1, 11. 
59-61). 
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Regarding dependent claim 3, W3C discloses different time compressed versions (p. 26, ex. 2). 
Regarding dependent claim 4, W3C discloses versions having different resolutions (p. 26, ex. 
4). 

Regarding dependent claim 5, W3C discloses an identifier for each of the plurality of media 
streams (p. 19, "src")- 

Regarding dependent claim 6, W3C discloses the identifier is a URI (p. 19, "src") equivalent to 
a URL. 

Regarding independent claim 7, W3C discloses an SMIL document that correlates an 
annotation with different versions of the same multimedia content as described in the rejection of 
claim 1 . It is the position of the Office that a data structure is equivalent to an XML document 
such as a SMIL. W3C does not disclose storing the annotation. deVries discloses a storage 
device to store the annotation (FIG. IB, 26). It would have been obvious to combine deVries 
and W3C to store the annotation so that it is correlated through the data structure by storing the 
annotation in the storage device at the location referenced by the URI in the <text/> tag. 
Regarding dependent claim 8, W3C discloses a version list identifying multiple different 
versions (p. 26, ex. 4). 

Regarding independent claims 10, 25 and 28, W3C teaches a data field representing an 
annotation corresponding to multimedia content, and a data field correlated so that the field 
identifies a plurality of different versions of the multimedia content to which the annotation 
corresponds (p. 26, ex. 4). 

Regarding dependent claims 11 and 26, W3C teaches that identifiers are intended to be 
included at the ellipses in example 4, on page 26. 
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Regarding dependent claim 9, 12, and 27, W3C teaches that URI's, equivalent to URL's are 
intended to be included at the ellipses in example 4, on page 26. 

Regarding independent claim 20, W3C discloses receiving an indication of the version of 
media content (p. 18, 11 1-3). W3C also discloses that annotations are corresponding to a 
plurality of different versions of the media content (p. 26, ex. 4). W3C is silent on providing the 
annotations to he the user. DeVries discloses providing presenting media and annotations to a 
user (col. 19, 11. 29-42). It would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine W3C and deVries as the markup language of W3C was intended to 
describe how media should interact with each other when it is displayed. 
Regarding dependent claim 22, while neither deVries nor W3C explicitly teach comparing 
identifiers, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to compare the identifiers to find out what media the version was a part of 
Regarding dependent claim 23, the computer programs for executing the methods of claims 20 
are rejected under the same rationale. 

7, Claims 13-15, 18-19, and remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over W3C in view of Logan et aL (USPN 5735216— patented 3/24/1998). 
Regarding independent claim 13, W3C teaches associating an annotation with a set of media 
streams, as set forth in the rejection of claim 1 above, however, is silent as to the creation of the 
annotations. Logan teaches receiving a user request to create a new annotation (col. 32, 11. 53- 
56). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
include a method of creating a new annotation, as it would be allow easier addition of 
annotations. 
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Regarding dependent claim 14, W3C does not explicitly disclose associating anew record with 
a previously generated record W3C does disclose annotations in <par> and <seq> tags (sees. 
4.2. 1-.4. 1 .2). Upon the creation of new annotation as recited in claim 13, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to associate the annotation 
within the tags to allow the annotation to be synched with the media stream. 
Regarding dependent claim 15, it would have been inherent to identify the streams before 
associating anything with them. 

Regarding dependent claim 18, W3C discloses a record of the set of media streams (p.26, 
ex.4). It would have been obvious to one of ordinary skill in the art at the time of the invention 
to generate and store this record so that it can be read later. 

Regarding dependent claim 19, the computer readable media for executing the methods of 
claims 13 are rejected under the same rationale. 

8. Claims 16-17 and 24 remain rejected under 35 U.S.C. 103(a) as being unpatentable 
over W3C and Logan as applied to claim 13 above, and further in view of deVries. 
Regarding dependent claim 16, while there is no explicit mention of multimedia servers in 
W3C and Logan, W3C does however disclose that both the annotations and media are referenced 
by a URI (p. 19, "src"), which were well-known to refer to servers. deVries et al. (deVries) 
discloses that media and annotations can be stored on servers (FIG IB, 22 and 26), the 
annotation server inherently having a database of annotations. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine deVries and 
W3C in order to effectively store the annotations (col. 14, 11. 5-8), media (col. 4, 11. 34-08), and 
relations between the two (FIG. 7). 
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As the annotations would have been stored on the server, it would have been obvious to 
communicate with server and identify the streams before associating anything with them. 
Regarding dependent claim 17, W3C discloses a set of identifiers, each identifier uniquely 
identifying one of the sets of streams (p. 26, ex. 4). 

Regarding dependent claim 24, W3C is silent on providing annotations to a user. DeVries 
discloses providing presenting media and annotations to a user (col. 19, 11. 29-42). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine W3C, 
Logan and deVries as the markup language of W3C was intended to describe how media should 
interact with each other when it is displayed. 

9. Claim 21 remains rejected under 35 U.S.C. 103(a) as being unpatentable over W3C 
and deVries as applied to claim 20 above, and further in view of Media Weaver, by Sha Xin 
Wei (6/2/1997). 

Regarding dependent claim 21, deVries discloses that annotations are associated with a time 
(col. 16, 11. 42-54). Both W3C and Logan are silent as to converting media. Xei discloses 
converting media (p. 12 bullet 3). It would have been obvious to one of ordinary skill in the art at 
the time of the invention modify Xei into W3C and Logan so that the annotations would still be 
presented in the same context as in the original media. 

Response to Arguments 
1 1 . Applicants arguments filed 1/27/2004 have been fully considered but they are not 
persuasive. 

Regarding Applicants remarks on §101 rejections: 
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Applicant has objected to the lack of a legal basis for holding a non- functional data structure to 
non-statutory. The Office has expanded it's explanation of the rejections under § 101 in 
paragraph 4 of the instant action. 

Regarding Applicant's remarks on claim 1, starting on p. 26: 

AppHcant argues that the text tag does not qualify as annotations. The Office disagrees. The 
text is contained with a <par> tag (§ 4.2. 1), with the <switch> block. Within the switch block, as 
noticed in the example's text, are "alternative parts designed for screens with different 
resolutions, and bit-depths.*' The ellipses at 6, 9, and 12, are clearly meant to be the alternative 
parts. These are the different versions of multimedia content. They would be represented by one 
of the tags disclosed in §4.2.3. Both those tags, and <text> when contained within a <par> can 
be associated to particular time frames (p. 12). While W3C indeed does not make mention of 
the word "annotation," it is still the position of the Office, that text associated with media is an 
annotation. As the text is assigned to a particular "temporal location in the multimedia 
presentation," it is equivalent to an annotation as defined in the specification, (page 2, line 19 - 
page 3, hne 7), even meeting the specific example of "temporal location." The title of section 
that Applicant points out is referring to the elements within the switch. The ellipses within the 
text tag do not only indicate that the source is arbitrary, but also that any attribute could be 
contained within, such as the ones hsted in §4.2.3. 

Applicant also again alleges the Office has admitted in the previous responses that the 
W3C is silent with respect to annotations. That is not an accurate assessment of what is said in 
this and the previous actions. The Office merely acknowledged that the word "annotation" was 
not used. Just because the same claim language is not used, does not mean that W3C is absent of 
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annotations. The Office has explained in the rejection and with the above argument why W3C 
does in fact teach annotations. 

In regard to the remarks on the deVries patent, again the Applicant merely recites the 
claims and alleges the source does not teach the claims. This does not constitute a valid 
argument because they amount to a general allegation that the claims define a patentable 
invention without specifically pointing out how the language of the claims patentably 
distinguishes them fi-om the references. Additionally, though deVries has not been relied on for 
teaching that an annotation is associated with muhiple versions of multimedia content, Applicant 
alleges that it does not contain such a teaching. Despite the one section of deVries the Applicant 
quotes, the entire specification, exemplified by Fig. 7, teaches annotation is associated with 
multiple versions of multimedia content. The above rejections have all been rejected under 35 
U.S.C. 103 under combinations of references. In response to appHcant's argument that deVries 
does not teach the entire claims, the test for obviousness is not whether the features of a 
secondary reference may be bodily incorporated into the structure of the primary reference; nor 
is it that the claimed invention must be expressly suggested in any one or all of the references. 
Rather, the test is what the combined teachings of the references would have suggested to those 
of ordinary skill in the art. See/n re Keller, 642 F.2d413, 208 USPQ 871 (CCPA 1981). 
Regarding dependent claim Applicant's remarks regarding claim 13, 
Applicant again alleges that W3C does not teach annotations. This argument has been addressed 
in the discussion of claim 1 above. 
Regarding Applicant*s remarks on claims 19 and 23: 
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Applicant is claiming a computer readable medium that executes the method of claim 13. No 
further "meaningful legal analysis" is deemed necessary as the claim is limited by the same 
limitations as claim 13, and therefore, as stated in the original rejection, is "rejected under the 
same rationale." Claim 23 has been treated in a similar manner. 
Regarding Applicant's remarks on the MPEP: 

Applicant merely summarizes the obviousness requirements. These remarks are considered to be 
non-persuasive as Applicant is not pointing out which of these criteria the Office has not met, 
nor which parts of the rejections these are relevant to, nor how these specifically apply to the 
Applicant's claims. 

Regarding Applicant's remarks regarding claims 2-6, 8, 9, 11, 12, 14-19, 21-24, and 26-27: 

Applicant alleges these claims are allowable in view of their base claims being allowable. As the 

rejections have been maintained and no substantive arguments have been provided as to the 

dependent claims patentability, the claims remain rejected. 

Regarding Applicant's remarks on 18-19, regarding Examiner's Response: 

Applicant again alleges the previous Examination was deficient without citing ANY specific 

examples. The Office sees no deficiencies its prior response to AppKcant's previous arguments. 

The Applicant makes no allegations of what arguments in particular were deficient. Applicant 

alleges repeatedly that the Office has not addressed all arguments, and various deficiencies 

within the arguments without providing any indication of what in particular arguments are 

deficient. As such the Office can not respond to such allegations. 

Applicant alleges there was no response to the arguments with respect to anticipation as there are 
no anticipation rejections in the previous Office Action. Additionally the cited portion of the 



Application/Control Number: 09/396,984 Page 13 

Art Unit: 2178 

MPEP, specifically the portion labeled "Examiner's Note," in addition to being located nowhere 
near the previous cited portion, has been taken completely out of context. The Note relates to a 
form paragraph used by the Office, which has not been used in this or the previous rejections. 

Conclusion 

10. TfflS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 1 36(a) will be calculated from the maihng date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Adam M Queler whose telephone number is (703) 308-5213. 
The examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather R Hemdon can be reached on (703) 308-5186. The fax phone number for 
the organization where this appHcation or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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